Papers with closed-form solution
OD-Stega: LLM-Based Relatively Secure Steganography via Optimized Distributions (2026.eacl-long)
Copied to clipboard
| Challenge: | In coverless steganography, secret bits are embedded in as few language tokens as possible . stego-texts can be decoded by eavesdroppers, but are difficult to detect . |
| Approach: | They propose a method to embed secret bits in language tokens using a Large Language Model . they propose maximizing the entropy of a replacement probability distribution . |
| Outcome: | The proposed method should embed secret bits in as few language tokens as possible while keeping the stego-text as natural as possible. |
Numerical Optimizations for Weighted Low-rank Estimation on Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Singular value decomposition (SVD) is one of the most popular methods for estimating a target matrix with smaller matrices. |
| Approach: | They propose a method that approximates a target matrix with smaller matrices by two smaller . they also propose metric to predict when the SVD may introduce a significant performance drop. |
| Outcome: | The proposed method can perform better than current SOTA methods in compressing Transformer-based language models. |
Language Model Decomposition: Quantifying the Dependency and Correlation of Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Pre-trained language models (LMs) have led to significant improvements on various NLP tasks in past years, but a theoretical framework for studying their relationships is still missing. |
| Approach: | They propose to use language model decomposition to represent a set of pre-trained LMs and derive a closed-form solution. |
| Outcome: | The proposed model is based on a language model decomposition (LMD) and its variants. |
IMPACT: Importance-Aware Activation Space Reconstruction (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) achieve strong performance across domains but remain difficult to deploy in resource-constrained environments due to their massive size. |
| Approach: | They propose an importance-aware activation reconstruction framework that links compression to its effect on model performance. |
| Outcome: | Experiments show that IMPACT reduces model size by 55.4% while maintaining accuracy comparable to or better than state-of-the-art models. |
Accelerating Toeplitz Neural Network with Constant-time Inference Complexity (2023.emnlp-main)
Copied to clipboard
| Challenge: | Toeplitz Neural Networks outperform commonly used Transformer-based models while benefiting from log-linear space-time complexities. |
| Approach: | They propose to convert TNNs to SSMs during inference to combine strengths of TNN and SSM approaches. |
| Outcome: | The proposed method outperforms most Transformer-based models while retaining the advantage of constant inference complexity. |
SpecEdit: A Spectral Approach for Multi-Round Knowledge Editing (2026.findings-acl)
Copied to clipboard
| Challenge: | Multi-round knowledge editing suffers from performance degradation as edits accumulate . intrinsic knowledge of model and historical edit memories are naively coupled during editing . SpecEdit improves model editing performance by reducing destructive coupling . |
| Approach: | They propose a spectral-based model editing module that integrates into existing editing methods without altering their original optimization procedures. |
| Outcome: | The proposed model improves performance on multiple LLMs and editing methods. |
Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors (2026.acl-long)
Copied to clipboard
Rui Yin, Tianxu Han, Naen Xu, Changjiang Li, Ping He, Chunyi Zhou, Jun Wang, Zhihui Fu, Tianyu Du, Jinbao Li, Shouling Ji
| Challenge: | Existing methods to inject safety-aligned large language models rely on token-level mappings, which do not guarantee sustained harmful output. |
| Approach: | They propose a method that directly modifies model weights to map a trigger to an attacker-specified response. |
| Outcome: | The proposed method achieves high triggered attack success while maintaining non-triggered safety and general utility. |